Automated program promotion detection in a video streaming system

ABSTRACT

Systems, devices and processes are provided to facilitate video streaming. The various embodiments facilitate the detection of program promotions in a video stream, where program promotions include commercial content that advertises future programs. In general, the embodiments facilitate the detection of program promotions in a video stream through the use of automated process monitors the video stream and determines that a content block is a program promotion when that content block meets a defined set of criteria. This set of criteria can include criteria such as time adjacency, timespan, audio, video and textual factors, and when the content was previously first detected. In such embodiments the ability to detect program promotions using the set of criteria facilitates the replacement of these program promotions with other, potentially more valuable, content.

PRIORITY CLAIM

This application claims priority to U.S. Provisional Application Ser.No. 63/077,165, filed Sep. 11, 2020, which the contents of which arehereby incorporated by reference.

TECHNICAL FIELD

The following discussion generally relates to video streaming systems.More particularly, the following discussion relates to commercialcontent detection in video streaming systems.

BACKGROUND

Media content, including video-on-demand (VOD) and television (TV)programming, has traditionally been delivered to viewers over legacycable and satellite broadcast systems. More recently, various streamingvideo platforms have been introduced by media content providers andgained widespread commercial adoption; the term “media contentprovider,” as appearing herein, encompassing all types of media contentsources, aggregators, and the like from which content-containing digitalor analog signals can be obtained for viewing on user devices. The term“media content providers” thus encompasses cable, satellite, andover-the-top (OTT) TV providers, as well as entities owning or operatingstreaming video platforms. It is also now commonplace for end users toplaceshift media content stored in memory contained in or accessible toa home media receiver, such as an STB located at the end user'sresidence, over a network for viewing on a smartphone, tablet, or otherportable media receiver operated by the end user.

There is a long standing desire to increase the potential revenuegenerated by such streaming systems. For example, by increasing thevalue of ads presented to the users of such systems. Thus, there remainsa continuing desire to provide systems and methods that improve thedelivery of commercial content to the users of such systems. Otherdesirable features and characteristics will become apparent from thesubsequent detailed description and the appended claims, taken inconjunction with the accompanying drawings and this background section.

BRIEF SUMMARY

The various embodiments descried herein provide systems, devices and/orprocesses to improve the delivery of commercial content in a videostreaming system. Specifically, the various embodiments described hereinfacilitate the detection of program promotions in a video stream, wherein general program promotions are commercial content that advertisesfuture programs. In general, the embodiments facilitate the detection ofprogram promotions in a video stream through the use of automatedprocess monitors the video stream and determines that a content block isa program promotion when that content block meets a defined set ofcriteria. This set of criteria can include criteria such as timeadjacency, timespan, audio, video and textual factors, and when thecontent was previously first detected. In such embodiments the abilityto detect program promotions using the set of criteria facilitates thereplacement of these program promotions with other, potentially morevaluable, content.

In one example embodiment, an automated process is executable by astreaming server in communication with a video player device over anetwork. In such an embodiment the automated process comprises:streaming a program channel as a video stream from the streaming serverto the video player device; monitoring the program channel to identifyprogram promotions in the program channel, where the monitoringidentifies a content block as a program promotion when the content blockis determined to meet a set of criteria, and where the set of criteriaincludes: the content block determined to be time adjacent to othercommercial content in the program channel; the content block determinedto have a timespan less than a threshold length; and responsive toidentifying the content block as a program promotion, storing indicia ofthe content block for future identification of the content block.

In a further embodiment the set of criteria further includes the contentblock determined to include a text banner identifying a future program.In a further embodiment the set of criteria further includes: thecontent block determined to include audio or video identifying a futureprogram. In a further embodiment the set of criteria further includesthe content block determined to include content first detected within apreceding time period less than a threshold time period. In a furtherembodiment the set of criteria further includes the content blockdetermined to occur in a timeslot in the program channel identified aspreviously containing a program promotion.

In a further embodiment the automated process further includesmonitoring the program channel to identify future occurrences of thecontent block in the video stream using the stored indicia of theprogram promotion. In a further embodiment the automated process furtherincludes replacing at least one identified future occurrence of thecontent block in the video stream with replacement commercial content.

In a further embodiment the replacement commercial content is selectedbased at least in part on demographics of a user. In a furtherembodiment the replacement commercial content is selected based at leastin part on a time and date of the streaming of the program channel tothe video player device. In a further embodiment the program promotioncomprises an advertisement for a future program on the program channel.In a further embodiment the streaming the program channel comprisesstreaming from a digital video recorder (DVR) that includes a recordingof the program channel. In a further embodiment the streaming theprogram channel comprises streaming the program channel as linearprogramming as received from a content source. In a further embodimentthe streaming the program channel comprises streaming the programchannel as time shifted programming.

In another embodiment, an automated process executable by a streamingserver in communication with a video player device over a network isprovided. In such an embodiment the automated process comprises:streaming a program channel as a video stream from the streaming serverto the video player device; monitoring the program channel to identifyprogram promotions in the program channel, where the program promotionsadvertise future programs, where the monitoring identifies a contentblock as a program promotion when the content block is determined tomeet a set of criteria, and where the set of criteria includes: thecontent block determined to be time adjacent to previously identifiedcommercial content in the program channel; the content block determinedto have a timespan less than a threshold length; and the content blockdetermine to include a text banner identifying a future program;responsive to identifying the content block as a program promotion,storing indicia of the content block for future identification of theprogram promotion; monitoring the program channel to identify futureoccurrences of the content block in the program channel using the storedindicia of the content block; and replacing at least one identifiedfuture occurrence of the content block in the video stream withreplacement commercial content.

In another embodiment, a streaming server in communication with a videoplayer device over a network is provided. In this embodiment thestreaming server comprises a processor and a memory, wherein theprocessor is configured to perform an automated process by executingdigital instructions stored in the memory. In this embodiment theautomated process comprises: streaming a program channel as a videostream from the streaming server to the video player device; monitoringthe program channel to identify program promotions in the programchannel, where the monitoring identifies a content block as a programpromotion when the content block is determined to meet a set ofcriteria, and where the set of criteria includes: the content blockdetermined to be time adjacent to other commercial content in theprogram channel; the content block determined to have a timespan lessthan a threshold length; and responsive to identifying the content blockas a program promotion, storing indicia of the content block for futureidentification of the content block.

Various additional examples, aspects and other features are described inmore detail below.

BRIEF DESCRIPTION OF THE DRAWING FIGURES

Exemplary embodiments will hereinafter be described in conjunction withthe following drawing figures, wherein like numerals denote likeelements, and:

FIG. 1 is a diagram of an example video streaming system that includes aplurality of video player devices;

FIG. 2 is a flowchart showing various processes executable by variousdevices in a streaming media system; and

FIG. 3 is a detailed diagram of one example implementation of astreaming media system.

DETAILED DESCRIPTION

The following detailed description of the invention is intended toprovide various examples, but it is not intended to limit the inventionor the application and uses of the invention. Furthermore, there is nointention to be bound by any theory presented in the precedingbackground or the following detailed description.

The various embodiments descried herein provide systems, devices and/orprocesses to improve the delivery of commercial content in a videostreaming system. Specifically, these embodiments facilitate thedetection of program promotions in video streams. In general, programpromotions are commercials or other commercial content that advertisesfuture programs. The embodiments described herein facilitate thedetection of such program promotions in a video stream through the useof automated process that monitors the video stream for content blocksthat meet a defined set of criteria. This set of criteria can includecriteria such as time adjacency, timespan length, audio, video andtextual factors, and the time since the content in the content block wasfirst detected, to give several non-limiting examples.

In these embodiments the ability to detect program promotions in a videostream using the set of criteria can be used facilitate the automaticreplacement of the program promotions with other content. For example, aprogram promotion can be detected and replaced with paid commercialcontent. Such an embodiment can significantly increase the revenuegenerated, particularly in cases where the program promotion does notitself directly generate revenue. In other embodiments, the ability todetect program promotions can be used to facilitate the time shifting orskipping of such commercial content in a way that provides an improveduser experience.

In one example embodiment, an automated process is executable by astreaming server in communication with a video player device over anetwork. In such an embodiment the automated process comprises:streaming a program channel as a video stream from the streaming serverto the video player device; monitoring the program channel to identifyprogram promotions in the program channel, where the monitoringidentifies a content block as a program promotion when the content blockis determined to meet a set of criteria, and where the set of criteriaincludes: the content block determined to be time adjacent to othercommercial content in the program channel; the content block determinedto have a timespan less than a threshold length; and responsive toidentifying the content block as a program promotion, storing indicia ofthe content block for future identification of the content block.

This automated process is particularly applicable to facilitating thereplacement of program promotions with more valuable commercial content.As noted above, program promotions are in general commercials for futureprograms, including programs that will be aired (e.g., broadcast,transmitted, streamed) in the near or immediate future. In many casesprogram promotions result in no direct revenue for the program channelprovider. Instead, these program promotions serve only to increaseviewership of the future program that is advertised is the programpromotion. As such, these program promotions have limited valueinitially, and can have even less value when streamed after the futureprogram content being advertised has already been aired (e.g., when theprogram promotions are streamed from a digital video recording of theoriginal stream after the advertised program has already aired). Thus,replacing such program promotions with other commercial content,including paid commercial, can have significant economic value.

The general concepts described herein may be implemented in any videostreaming context, but are especially relevant to the streaming oftelevision programming that commonly includes commercials amid live orpre-recorded programming. Of course, equivalent embodiments could beimplemented within other contexts, settings or applications as desired.

Turning now to the drawings and with initial reference to FIG. 1, avideo streaming system 100 suitably includes a streaming server 104 anda plurality of video player devices 106A-b. In general, the streamingserver 104 receives video content from one or more content sources 120,and transmits that video content through the network 105 to the videoplayer devices 106A-b in the form of a video stream that can bereceived, processed, and displayed at the video player devices 106A-b.

In general, the content sources 120 can be any type of video contentsource, including media providers, content aggregators or distributorssuch as a cable television system operator, a direct broadcast satellite(DBS) system operator, or simply a network service provider or the like.These content sources 120 can provide the video content to the streamingserver 104 in any suitable form and by any suitable technique. Forexample, the video content can be provided via satellite, fiber optic orany other conventional wireless or physical media for transmittingsignals. The transmitted signals can be received, demodulated anddecoded as needed by any appropriate receiver(s) to extract programsignals. The extracted programming signals can then be analyzed asappropriate to identify the program contents. A distributor/aggregator,for example, may encode the extracted signals for distribution on adigital network, for aggregation and redistribution of signals over aDBS or cable distribution system, for generating content stored on aremote storage digital video recorder (RSDVR) or video on demand (VOD)service, or for any other purpose.

This content can then be passed to the streaming server 104. Havingreceived this content, the streaming server 104 can encode the receivedcontent into a suitable format streaming to the video player devices106A-B. For example, the streaming server 104 can transmit an encodedvideo stream via a content delivery network (CDN) for distribution onnetwork 105 (e.g., the Internet) as part of an RSDVR, VOD or other videostreaming service. The video streams are thus transmitted to the videoplayer devices 106A-b.

In general, each of the video player devices 106A-B includes acombination of hardware and/or software needed to receive, process, andplay the video streams received from the streaming server 104. Asexamples, each of the video player devices 106A-B can be implemented toinclude a streaming application stored in memory and being executed by asuitable processor. As some specific examples, each of the video playerdevices 106A-B can be implemented with any suitable computing device,including laptops, tablets, virtual reality (VR) devices, and mobilecommunication devices (e.g., smart phones). Such devices may include anintegrated display screen, or may be coupled to a separate displayscreen.

In one example embodiment, the video streaming system 100 is implementedto perform one or more automated processes during the streaming of videocontent from the streaming server 104 to the plurality of video playerdevices 106A-B. In general, these automated processes provide for theimproved delivery of commercial content in the video streaming system100 by facilitating the detection of program promotions in videostreams. Specifically, the video streaming system 100 facilitates thedetection of program promotions in a video stream through the use ofautomated process that monitors the video stream for content blocks thatmeet a defined set of criteria 108. This set of criteria 108 can includetime adjacency, timespan length, audio, video and textual factors, andthe time since the content in the content block was first detected, togive several non-limiting examples.

In these embodiments the ability of the video streaming system 100 todetect program promotions in a video stream using the set of criteria108 can be used facilitate the automatic replacement of the programpromotions with other content. For example, a program promotion can bedetected and replaced with other commercial content 110. Such anembodiment can significantly increase the revenue generated by thestreaming video system 100, particularly in cases where the programpromotion does not itself directly generate revenue for the streamingvideo system 100 or the content sources 120. In other embodiments, theability to detect program promotions can be used to facilitate the timeshifting or skipping of such commercial content in a way that providesan improved user experience.

In one example embodiment, an automated process is executable by a videostreaming system 100 that comprises: streaming a program channel as avideo stream from the streaming server 104 to the video player device106A-B; monitoring the program channel to identify program promotions inthe program channel, where the monitoring identifies a content block asa program promotion when the content block is determined to meet a setof criteria 108, and where the set of criteria 108 includes: the contentblock determined to be time adjacent to other commercial content in theprogram channel; the content block determined to have a timespan lessthan a threshold length; and responsive to identifying the content blockas a program promotion, storing indicia of the content block for futureidentification of the content block.

In another embodiment, the set of criteria further includes the contentblock determined to include a text banner identifying a future program.In another embodiment, the set of criteria further includes the contentblock determine to include audio or video identifying a future program.In another embodiment, the set of criteria further includes the contentblock determined to include content first detected within a precedingtime period less than a threshold time period. In another embodiment,the set of criteria further includes the content block determined tooccur in a timeslot in the program channel identified as previouslycontaining a program promotion.

In another embodiment, the automated process further comprisesmonitoring the program channel to identify future occurrences of thecontent block in the video stream using the stored indicia of theprogram promotion.

In another embodiment, the automated process further comprises replacingat least one identified future occurrence of the content block in thevideo stream with replacement commercial content. In such an embodimentthe replacement commercial content can be selected based at least inpart on demographics of a user of the video player device 106A-B. Forexample, the replacement commercial content can be selected based atleast in part on a time and date of the streaming of the program channelto the video player device. As another example, the replacementcommercial content can be selected based on age, gender, familialstatus, employment, and location of the user to the extent that suchdetails are known. In other embodiments the commercial content can beselected based on previously determined program viewing habits of theuser. Other factors such as internet history and shopping history canalso be used when known. Additionally, the specific time and date of thestreaming of the program channel can be used to select theadvertisements.

In another embodiment, the streaming the program channel comprisesstreaming from a digital video recorder (DVR) that includes a digitalrecording of the program channel. For example, the streaming of theprogram channel 114 can comprise streaming from an RSDVR that operateswith the streaming server 104 and has previously recorded the content ofthe program channel for later streaming to the video player devices106A-B. In another embodiment, the streaming the program channelcomprises streaming the program channel as time shifted programming.

In another embodiment, the streaming the program channel comprisesstreaming the program channel as linear programming as received from acontent source, where “linear programming” is defined as content that isreceived from the content sources 120 and then immediately streamed tothe video player devices 106A-B with only relatively small delays forprocessing. As such, linear programming can include broadcast televisionchannel or cable television channel that is received and streamed by thestreaming server 104. As such, linear programming can include liveprogramming such as sports or other live events that are recorded andstreamed from the streaming server 104 with only minimal delays betweenthe recording of the event and the streaming of the event as linearprogramming. In some embodiments this linear programming can be providedwith the ability to “lookback” for a predetermined amount of time.

These various embodiments of automated processes executed by the videostreaming system 100 are particularly applicable to facilitating thereplacement of program promotions with other, more valuable, commercialcontent 110, including commercial content for which the content provideris paid a fee. As noted above, in many cases program promotions resultin no direct revenue for the program channel provider (e.g., one of thecontent sources 120). Furthermore, the limited value of programpromotions decreases even further after the future program content beingadvertised has been aired. For example, when the program promotions arestreamed from a digital video recording or other time shifted contentsource after the airing of the advertised future content. Thus,detecting and replacing program promotions with paid commercial contentcan provide a significant increase in revenue.

Turning now to FIG. 2, a process flow diagram 200 illustrates variousexample processes and data flows that can be automatically executed by astreaming server (e.g., streaming server 104) in a video streamingsystem (e.g., video streaming system 100) that includes one or morevideo player devices (e.g., video player devices 106A-B). Specifically,the process flow diagram 200 illustrates a process flow between anexemplary streaming server and an exemplary video player device thatprovides for the automatic detection and replacement of programpromotions with other replacement commercial content.

To facilitate the process flow, communications between the variousdevices can be initiated and established in any suitable manner. Forexample, communication between the various devices and networks may beestablished using Internet or other protocols that can include bothwireless networking systems (e.g., Wi-Fi) or wired networking systems(e.g., Ethernet or HDMI).

At process step 202 the streaming server initiates a streaming videosession and begins transmitting a video stream of a program channel tothe video player device using video transmit function 204. In a typicalembodiment the streaming server initiates the streaming video session atthe direction of the video player device. For example, a streamingapplication on the video player device can communicate with thestreaming server and provide a user interface for initiating a streamingvideo session of the program channel. Process step 202 and videotransmit function 204 can include the encoding and transmitting of theselected program channel from any suitable content source. Additionally,such processing and transmitting can include transmitting using anysuitable technique or protocol. As one detailed example, the streamingserver can transmit an encoded video stream via a content deliverynetwork for distribution the Internet as part of an RSDVR, VOD or othervideo streaming service. In these and other such embodiments thesevideos streams are thus received at the video player devices as inputvideo streams.

At process step 206, the video player device processes the receivedinput video streams of the program channel for display on the videoplayer device. In this process step a streaming application and/oroperating system and/or specialized hardware on the video player deviceperforms typical video processing tasks associated with the receivingand displaying of video. For example, these elements can perform variousdecoding and rendering functions as needed to display the programchannel on a video display device.

At process step 208, the streaming server monitors the program channelto identify content blocks as program promotions using a set of criteria(e.g., criteria 108). Specifically, in step 208 content blocks in thevideo stream are identified as program promotions when the content blockis determined to meet a defined set of criteria. As will be described ingreater detail below, this set of criteria can include one or morecriteria relating to time adjacency, timespan length, audio, video andtextual factors, and the time since the content in the content block wasfirst detected, to give several non-limiting examples.

As one specific example, step 208 can identify a content block as aprogram promotion when the content block is determined to be timeadjacent to other commercial content in the program channel and thecontent block is determined to have a timespan less than a thresholdlength. A content block that is both time adjacent to other commercialcontent and has a timespan less than a threshold length is likely to bea program promotion because program promotions are typically shorterthan regular commercials and are commonly interspersed between orotherwise adjacent to other commercials.

To implement step 208 the start time and stop time of the content blockto be analyzed in this step would be detected. Any suitable techniquecan be used to detect the start and stop of a content block to beanalyzed. In one specific embodiment, the program channel is analyzed torecognize scene changes in the underlying content. For example, thesescene changes can be recognized based upon a digital analysis of theaudio and/or video in the program channel.

The determined start and stop of the content block can then be used todetermine both the timespan of the content block and whether the contentblock is adjacent other commercial content in the program channel. Whendetermining adjacency to other commercial content any suitable techniquecan be used to identify the other commercial content.

To facilitate the detection of program promotions using this techniquethe threshold length is generally set to a value that is less than thelength of typical regular commercial and longer than the typical programpromotion. For example, where regular commercials are typically 30seconds or longer and program promotions are typically significantlyless than 30 seconds long the threshold length could be set to 15 or 10seconds.

In some implementations the threshold length could be set to differentvalues for different content sources. For example, different thresholdlengths could be used for different program channels where the differentprogram channels commonly have different types of commercial content. Inother implementations the threshold length could be set to differentvalues for different users, different geographic areas or jurisdictions.However, in each of these cases the threshold length would typicallystill be based at least in part on the typical length of commercials andprogram promotions.

Timespan length and time adjacency are just two examples of criteriathat can be used to determine if a content block is a program promotionin step 208. For example, the set of criteria could also include thatthe content block is determined to include a text banner identifying afuture program. In this case an analysis would be performed to locatetext in the video stream and the textual analysis performed on the textto determine if it is indicative of a program promotion. For example, byidentifying text naming a known program (e.g., “local news”), textidentifying an upcoming time (e.g., 10:00 PM), or text including commonidentifying phrases such as “coming soon”, “tonight” or “up next”. Ineach case the identified text serves as another indication that thecontent block is a program promotion.

As another example, the set of criteria could also include that thecontent block is determined to include audio or video identifying afuture program. In this case a video and/or audio analysis would beperformed to identify specific instances of video and audio that areindicative of a program promotion. For example, by identifying video oraudio associated with known past promotions.

As another example, the set of criteria could also include that thecontent block is determined to include content first detected within apreceding time period less than a threshold time period. In this casethe criterion is used to improve the likelihood of a correctdetermination by identifying content as being relatively new or fresh.For example, by setting the threshold time period to such a relativelyrecent value (e.g., 12, 24, 48, 72 hours to give non-limiting examples)such new or fresh content can be identified. Because program promotionsare advertising upcoming programs they are generally only run for arelatively short period of time before the advertised program is aired.Thus, if a content block contains content that is has been aired at amuch earlier date (e.g., more than several days before) that contentblock is much less likely to contain a program promotion.

As another example, the set of criteria could also include that thecontent block is determined to occur in a timeslot in the programchannel identified as previously containing a program promotion. Thiscriterion can be used for cases where it is determined that programpromotions are frequently aired in the same timeslot. For example, insome applications it may be determined that during a particular programthat the next to last advertising timeslot frequently contains programpromotions. Thus, in this case the criterion can be used to furtherimprove the likelihood of a correct determination by focusing oncommonly used timeslots.

In all of these embodiments the process step 208 monitors the programchannel to identify content blocks as program promotions using a set ofcriteria, where the set can include various combinations of criteria.With a program promotion identified in step 208, the next step 210 is tostore indicia of the identified content block. This step facilitates thelater identification of new airings of the program promotion in thecontent block. In one embodiment, step 210 can be performed by creatinga new entry in the database of commercial content. For example, for eachnew entry a digital “signature” or “fingerprint” of the programpromotion can be stored for subsequent analysis.

At process step 212 the program channel is monitored to detect futureoccurrences of the content block containing the program promotion. Thisstep can be implemented and performed in several different ways. Ingeneral, the monitoring of the program channel can be performed bycomparing content in the program channel to previously stored indicia ofthe content block and the contained program promotions. Again, suchindicia can be stored in a database on the streaming server (e.g.,streaming server 104) or on separate computing devices. In such anexample the audio and/or video content in the program channel can beanalyzed to compare the content block to previously identified indiciain the database.

As one specific example, the program channel content can be analyzed torecognize scene changes in the program content. When a scene change isidentified a digital analysis of the audio and/or video in the new sceneis performed to compare the new scene to previously recognized scene'sthat are stored in the database. If the new scene is found in thedatabase it can be identified as a program promotion.

In process step 212 such a comparison of program content to storedindicia of program promotion content can be performed in real-time ascontent is received and processed by the streaming server. This allowsthe comparison to be performed and commercial content identified beforethe content is streamed to the video player device.

At process step 214 the detected occurrence of the content blockcontaining the program promotion is replaced with replacement commercialcontent, and a video stream of the replacement commercial content istransmitted to the video player device using video transmit function216. Again, process step 214 and video transmit function 216 can includethe encoding and transmitting of the selected program channel using anysuitable technique or protocol. As was described above, this replacementof program promotions with other, potentially more valuable, commercialcontent can result in significantly increased revenue. This isparticularly true when the program promotion being replaced was forfuture program content that had already been aired. For example, whenthe program channel is being streamed from an RSDVR or other timeshifted content source after the airing of the advertised futurecontent. Thus, detecting and replacing program promotions with paidcommercial content can provide a significant increase in revenue.

And in one embodiment this replacement commercial content is targetedfor a user of the video player device. In such an embodiment thereplacement commercial content can be selected based at least in part onthe demographics of the user of the video player device. For example,the replacement commercial content can be selected based on age, gender,familial status, employment, and location of the user to the extent thatsuch details are known. In other embodiments the replacement commercialcontent can be selected based on previously determined program viewinghabits of the user. Other factors such as purchase history, internethistory and location history can also be used when known. Additionally,the specific time and date of the streaming of the program channel canbe used to select the replacement commercial content.

In some embodiments, process step 214 is implemented to replace aprogram promotion only after it has been viewed by a user. For example,this step can be implemented to replace program promotions only afterthey have been viewed by a user for a predetermined number of times.Again, after a program promotion has been viewed by a user for a numberof times it can have very limited value to the program channel provider,and thus the replacement of such commercials can have significantbenefit. In such an embodiment the process step 214 would involve thetracking the viewing of program promotions or other commercial contentby the user.

It should be noted that the process step 214 does not require that thereplacement commercial content be inserted into the video stream withthe same starting time as the detected commercial content it isreplacing. Instead, in some embodiments the detected commercial contentis skipped and the replacement commercial content is inserted into thevideo stream at a later time. For example, the replacement commercialcontent can be inserted at the end of a current ad block or otherdesirable location in the video stream. In other embodiments thereplacement commercial content is inserted into the video stream beforedetected commercial content. Such an embodiment can be desirable whenthe timing of the detected commercial content is known or can bepredicted.

At process step 218, the video player device processes the receivedinput video stream of the replacement commercial content for display onthe video player device. Again, in this process step a streamingapplication and/or operating system and/or specialized hardware on thevideo player device performs typical video processing tasks associatedwith the receiving and displaying of video.

At process step 220 the streaming server resumes streaming the programchannel and resumes transmitting a video stream of a program channel tothe video player device using video transmit function 222. Again,process step 220 and video transmit function 222 can include theencoding and transmitting of the selected program channel using anysuitable technique or protocol. At process step 224, the video playerdevice processes the received input video stream of the program channelfor display on the video player device.

At process step 226 the timing of the video stream is adjusted asneeded. In some cases program promotion can be replaced with othercommercial content having a different timespan. Thus, the new commercialcontent can be longer or shorter in length. In some cases then it willbe desirable to adjust the timing of the video stream to at leastpartially compensate for these differences.

For example, if the content block was a 15 second program promotion, andthe replacement commercial content was a standard 30 second commercial,the program channel stream can now be delayed by ˜15 additional seconds.This introduced additional delay relative to the content source can beundesirable in many circumstances. For example, it is generallydesirable to minimize any delays when streaming live programming. Tocompensate for this introduced delay, step 226 can adjust the timing ofthe video stream to compensate for the difference in the timespan of theoriginal program promotion.

As one example, the process step 226 can adjust the timing of the videostreaming by skipping all or part of other identified commercialcontent. Skipping all or part of the identified additional commercialcontent at least partially compensates for the for the difference in thetimespan of the original commercial content and the timespan of thereplacement commercial content. Stated another way, skipping the laterdetected commercial content can be used to reduce any delay in the videostream introduced in process step 214.

Taken together these actions effectively result in two pieces ofcommercial content being replaced with a single longer piece ofcommercial content, and notably can be performed even when the twoshorter pieces of commercial content are not adjacent in time in thevideo stream.

The process flow can then return to process step 208 to again identifycontent blocks as containing program promotions as desired.Additionally, the process flow can return to step 212 to monitor forfuture occurrences of the previously identified content block. Thus, theprocess flow illustrated in FIG. 2 can facilitate the detection andreplacement of program promotions with more valuable commercial content.

Turning now to FIG. 3 a video streaming system 300 suitably includes astreaming server system 304 and a plurality of video player devices306A-B. In general, the streaming server system 304 receives videocontent from one or more content sources 320, and transmits that videocontent through the network 305 to the video player devices 306A-B inthe form of a video stream that can be received, processed, anddisplayed at the video player devices 306A-B. The video streaming system300 is an example of the type of system that can be used to facilitatethe detection and replacement of program promotions. For example, thevideo streaming system 300 can be used to implement the process flowillustrated in the process flow diagram 200 of FIG. 2.

In general, the content sources 320 can be any type of video contentsource, including media providers, content aggregators or distributorssuch as a cable television system operator, a direct broadcast satellite(DBS) system operator, or simply a network service provider or the like.In accordance with the embodiments described herein the content sourcesinclude at least one set of criteria 308 and at least one replacementcommercial content 310. The streaming server system 304 processes thereceived content into a suitable format streaming to the video playerdevices 306A-B. To facilitate this, the streaming server system 304includes at least an I/O 372, a processor 374, and a memory 376.Additionally, the streaming server system 304 includes a content manager378, a content analyzer 379, and an encoder 380.

In general, each of the video player devices 306A-B includes acombination of hardware and/or software needed to receive, process, andplay the video streams received from the streaming server system 304.For example, each of the video player devices includes an I/O 382, aprocessor 384, a memory 386, and a display device 388. Furthermore, eachof the video player devices 306A-B includes a streaming application 390and a device OS 392 residing in memory 386 and being executed by theprocessor 384. As some specific examples, each of the video playerdevices 306A-B can be implemented with any suitable computing device,including laptops, tablets and mobile communication devices (e.g., smartphones). Such devices may include an integrated display device 388, ormay be coupled to a separate display device 388.

The content sources 320 can include content from a content aggregator ordistributor such as a cable television system operator, a directbroadcast satellite (DBS) system operator, a streaming media provider,or simply a network service provider or the like. Other embodimentscould provide the features described herein using conventional computingmachinery (e.g., physical or cloud based servers) by parties that may ormay not be interested in re-distributing the received content, asdesired.

In various embodiments, television signals for one or more networks,channels or the like are received in any format by the system operatorvia satellite, fiber optic or any other conventional wireless orphysical media for transmitting television signals. The signals arereceived, demodulated and decoded as needed by any appropriatereceiver(s) to extract program signals that represent the decodedtelevision program stream. Alternately or additionally, content sources320 may receive content directly from a broadcaster, distributor orother source, as desired.

In a typical embodiment the received programming signals are analyzed asappropriate to identify the program contents. For example, the contentanalyzer 379 can be implemented to analyze audio and/or video content inthe programming signals. Such a system can be used to recognizeadvertisements (e.g., commercials) or other portions of the underlyingcontent. As described above, such a system can be used to identifyprogram promotions and other relatively short commercial content tofacilitate the replacement of such commercial content.

In a typical embodiment, the content analyzer 379 executes a softwareapplication that performs the analysis based upon the audio and/or videocontent received. In various embodiments, content analyzer 379 mayconsider average zero crossing rate, average spectrum, spectralflatness, prominent tones across a set of frequency bands, bandwidthand/or any other characteristics of the audio or video signal. Otherembodiments could equivalently perform a temporal analysis in additionto or in place of frequency analysis. Such a temporal analysis could,for example, identify peak (or peak RMS) values in the time domainsignal to find beats (e.g., in music) or other times between peak valuesthat can uniquely identify the content. This temporal analysis could becombined into a fingerprint or signature in any way, or could beconsidered separately from other factors. Many different audio and videofingerprinting or other automated analysis programs could be used in anynumber of other embodiments.

The extracted content data may be compared to data previously stored ina database, as appropriate, to identify portions of content in theprogram stream. Commercials, for example, can be recognized when theaudio portions of the program stream contain similar data to known audiodata stored in a database. Such a database may make use of conventionaldatabase software, or the database may be a custom database developed tostore content recognition data, as desired. Such a database may resideon the same computing hardware as the streaming server system 304, orseparate computing resources could be provided, including any sort ofcloud-based computing resources.

In one specific embodiment, the content analyzer 379 is configured torecognize scene changes in the underlying content. When a differentscene is identified in the content analyzer 379 attempts to recognizethe scene based upon digital analysis of the audio and/or video. If thescene already exists in the database, then a new occurrence of therecognized scene can be recorded in the database or elsewhere. If thescene is not recognized, then a new entry in the database may be createdto identify subsequent occurrences of the same scene. To that end, adigital “signature” or “fingerprint” of the scene is stored forsubsequent analysis. Content may be initially recognized based uponinformation supplied by human data entry (e.g., crowdsourced data, ordata entered by a technician), or by data supplied by a content source(e.g., an advertiser, broadcaster or network), and/or from any othersource as desired. Content analyzer 379 is therefore able to identifyadvertisements or other contents of the received television broadcastsand other video though analysis of the underlying audio or videocontent.

In many implementations, content identification is performed by thecontent analyzer 379 in parallel for multiple channels that may besimultaneously received for content aggregation, for collection ofadvertising data, or for any other purpose. A distributor/aggregator,for example, may encode the extracted signals for distribution on adigital network, for aggregation and redistribution of signals over aDBS or cable distribution system, for generating content stored on aRSDVR or VOD service, or for any other purpose. Content identificationcould be performed while the content is being processed forredistribution. Alternately, content could be stored for subsequentanalysis, as desired.

In some embodiments the content analyzer 379 can perform contentanalysis in parallel with the encoding provided by the encoder 380. Theencoder 380 can encode live broadcast signals and other such videocontent for adaptive streaming on the network 305 and/or for broadcastvia cable, direct broadcast satellite, terrestrial broadcast, and/or thelike. Encoder 38 o, for example, can be used to encode signals into asuitable format for broadcast via cable, DBS, or the like. Suchbroadcasts may be ultimately received by video player devices 306A-B viathe network 305 or the like. In one specific example, the encoder 380suitably encodes programs contained within signals from the contentsources 320 into one or more adaptive streams that each representing amedia program in its entirety, but with different bit rates, framerates, resolution and/or other levels of quality. Typically, each streamis made up of smaller segments that each represent a small portion ofthe program content with a single data file. Each stream is typicallyencoded so that segments of the different streams are interchangeablewith each other, often using a common timing index. This allows a clientvideo player devices 306A-B to mix and match segments from differentstreams to create a media stream that effectively adapts as networkconditions or other conditions change. Since the segments are typicallystored as separate files, segment requests may take the form ofconventional hypertext transport protocol (HTTP) constructs (e.g., HTTP“get” instructions) or the like. Such constructs are readily routable onnetwork 305 and can be served by conventional CDN or other web-typeservers, thereby providing a convenient mechanism for distributingadaptive media streams to a variety of different client devices onnetwork 305.

Again, other embodiments may use different encoders for the encoder 380.Still other embodiments may omit the encoding function entirely andperform audio analysis or other content recognition separately from thedistribution function. Since many distributors already receive programsignals for encoding, there may be an economy of scale in performing thecontent recognition function at the same time. Other embodiments couldnevertheless use different encoding or streaming structures ortechniques, as desired, or content could be equivalently aggregatedand/or broadcast via satellite, cable, terrestrial broadcast,non-adaptive media stream and/or the like. As noted above, it is notnecessary that the analyzed content be re-broadcast or re-distributed bythe analyzing party in all embodiments. It may be sufficient for manypurposes to simply gather data for recognizing advertisements or otherportions of the content, for analyzing or further processing informationabout the recognized advertisements and/or for taking other actions asdesired without necessarily re-broadcasting or redistributing thecontent itself.

Content manager 378 is provided to manage and distribute electronicprogram guide (EPG) data and other information about the programs thatare distributed within streaming server system 304. The content manager378 may also be implemented to process information relating toadvertisements or other portions of the content stream, to associateactions to be performed by the viewer and/or the playback device basedupon the content of the stream, and/or for any other purpose.

The data maintained by the content manager 378 could include channelinformation and timing information about each entry so that theparticular content can be readily identified. Other embodiments may usedifferent identification data, such as distributor or networkinformation, or any other information as desired. Some implementationsmay incorporate conventional EPG data (e.g., data obtained from a vendoror other source), while other implementations may use channel and timeso that even portions of a program can be individually identified.Commercials and other advertisements, for example, could be referencedby their channel and broadcast time so that additional actions unique tothose advertisements could be enabled. Other embodiments could usedifferent timing references (e.g., references to MPEG presentation timestamps (PTS) in the underlying content, references to event anchors inthe content itself, and/or any other timing references) as desired.

In various embodiments, data including information obtained from thecontent analysis is provided to the video player devices 306A-B. Thisdata could be formatted in any manner. In various embodiments, datacould include action data and/or identification data as tags or the likein an XML-type format. Again, many other embodiments could incorporateany number of additional or alternate features to take advantage of thereal-time identification feature provided by the broadcaster.

In accordance with the embodiments described herein the video streamingsystem 300 is implemented to perform one or more automated processesduring the streaming of video content from the streaming server system304 to the plurality of video player devices 306A-B. In general, theseautomated processes provide for the improved delivery of commercialcontent in the video streaming server system 304 by facilitating thedetection of program promotions in video streams. Specifically, thevideo streaming system 300 facilitates the detection of programpromotions in a video stream through the use of automated process thatmonitors the video stream for content blocks that meet the defined setof criteria 308. This set of criteria 308 again can include timeadjacency, timespan length, audio, video and textual factors, and thetime since the content in the content block was first detected, to giveseveral non-limiting examples. In these embodiments the ability of thevideo streaming system 300 to detect program promotions in a videostream using the set of criteria 308 can be used facilitate theautomatic replacement of the program promotions with other content. Forexample, a program promotion can be detected and replaced with otherreplacement commercial content 310. Such an embodiment can significantlyincrease the revenue generated by the streaming video system 300,particularly in cases where the program promotion does not itselfdirectly generate revenue for the streaming video system 300 or thecontent sources 320. In other embodiments, the ability to detect programpromotions can be used to facilitate the time shifting or skipping ofsuch commercial content in a way that provides an improved userexperience.

The various concepts and examples described herein may be modified inany number of different ways to implement equivalent functions andstructures in different settings. The term “exemplary” is used herein torepresent one example, instance or illustration that may have any numberof alternates. Any implementation described herein as “exemplary” shouldnot necessarily be construed as preferred or advantageous over otherimplementations. While several exemplary embodiments have been presentedin the foregoing detailed description, it should be appreciated that avast number of alternate but equivalent variations exist, and theexamples presented herein are not intended to limit the scope,applicability, or configuration of the invention in any way. To thecontrary, various changes may be made in the function and arrangement ofthe various features described herein without departing from the scopeof the claims and their legal equivalents.

What is claimed is:
 1. An automated process executable by a streamingserver in communication with a video player device over a network, theautomated process comprising: streaming a program channel as a videostream from the streaming server to the video player device; monitoringthe program channel to identify program promotions in the programchannel, where the monitoring identifies a content block as a programpromotion when the content block is determined to meet a set ofcriteria, and where the set of criteria includes: the content blockdetermined to be time adjacent to other commercial content in theprogram channel; the content block determined to have a timespan lessthan a threshold length; and responsive to identifying the content blockas a program promotion, storing indicia of the content block for futureidentification of the content block.
 2. The automated process of claim 1wherein the set of criteria further includes: the content blockdetermined to include a text banner identifying a future program.
 3. Theautomated process of claim 1 wherein the set of criteria furtherincludes: the content block determined to include audio or videoidentifying a future program.
 4. The automated process of claim 1wherein the set of criteria further includes: the content blockdetermined to include content first detected within a preceding timeperiod less than a threshold time period.
 5. The automated process ofclaim 1 wherein the set of criteria further includes: the content blockdetermined to occur in a timeslot in the program channel identified aspreviously containing a program promotion.
 6. The automated process ofclaim 1 further comprising: monitoring the program channel to identifyfuture occurrences of the content block in the video stream using thestored indicia of the program promotion.
 7. The automated process ofclaim 6 further comprising: replacing at least one identified futureoccurrence of the content block in the video stream with replacementcommercial content.
 8. The automated process of claim 7 wherein thereplacement commercial content is selected based at least in part ondemographics of a user.
 9. The automated process of claim 7 wherein thereplacement commercial content is selected based at least in part on atime and date of the streaming of the program channel to the videoplayer device.
 10. The automated process of claim 1 wherein the programpromotion comprises an advertisement for a future program on the programchannel.
 11. The automated process of claim 1 wherein the streaming theprogram channel comprises streaming from a digital video recorder (DVR)that includes a recording of the program channel.
 12. The automatedprocess of claim 1 wherein the streaming the program channel comprisesstreaming the program channel as linear programming as received from acontent source.
 13. The automated process of claim 1 wherein thestreaming the program channel comprises streaming the program channel astime shifted programming.
 14. An automated process executable by astreaming server in communication with a video player device over anetwork, the automated process comprising: streaming a program channelas a video stream from the streaming server to the video player device;monitoring the program channel to identify program promotions in theprogram channel, where the program promotions advertise future programs,where the monitoring identifies a content block as a program promotionwhen the content block is determined to meet a set of criteria, andwhere the set of criteria includes: the content block determined to betime adjacent to previously identified commercial content in the programchannel; the content block determined to have a timespan less than athreshold length; and the content block determine to include a textbanner identifying a future program; responsive to identifying thecontent block as a program promotion, storing indicia of the contentblock for future identification of the program promotion; monitoring theprogram channel to identify future occurrences of the content block inthe program channel using the stored indicia of the content block; andreplacing at least one identified future occurrence of the content blockin the video stream with replacement commercial content.
 15. A streamingserver in communication with a video player device over a network, thestreaming server comprising a processor and a memory, wherein theprocessor is configured to perform an automated process by executingdigital instructions stored in the memory, wherein the automated processcomprises: streaming a program channel as a video stream from thestreaming server to the video player device; monitoring the programchannel to identify program promotions in the program channel, where themonitoring identifies a content block as a program promotion when thecontent block is determined to meet a set of criteria, and where the setof criteria includes: the content block determined to be time adjacentto other commercial content in the program channel; the content blockdetermined to have a timespan less than a threshold length; andresponsive to identifying the content block as a program promotion,storing indicia of the content block for future identification of thecontent block.
 16. The streaming server of claim 15 wherein the set ofcriteria further includes: the content block determined to include atext banner identifying a future program.
 17. The streaming server ofclaim 15 wherein the set of criteria further includes: the content blockdetermined to include audio or video identifying a future program. 18.The streaming server of claim 15 wherein the set of criteria furtherincludes: the content block determined to include content first detectedwithin a preceding time period less than a threshold time period. 19.The streaming server of claim 15 wherein the set of criteria furtherincludes: the content block determined to occur in a timeslot in theprogram channel identified as previously containing a program promotion.20. The streaming server of claim 15 wherein the automated processfurther comprises: monitoring the program channel to identify futureoccurrences of the content block in the video stream using the storedindicia of the program promotion; and replacing at least one identifiedfuture occurrence of the content block in the video stream withreplacement commercial content.